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(54) Method and apparatus for real time scheduling in a satellite communications network 



(57) A method and apparatus are provided for 
scheduling, in real-time, the order in which data packets 
from a plurality of uplink channels 40-55 are organized 
in a downlink channel 60 of a satellite communications 
network 5. The satellite 1 0 includes uplink and downlink 
channels for conveying data packets over channels be- 
tween user terminals 15-30, ground stations 35 and oth- 
er user terminals. Queues 104- 110 in the satellite 10 col- 
lect data packets from uplinks 40-55 and output the data 



packets to the downlink 60 using a bandwidth that is dy- 
namically allocated. A scheduler 152 allocates the 
bandwidth to at least one queue 104-110 and continu- 
ously changes the amount of bandwidth allocated to 
each active queue 104-110 while the queue 104-110 is 
buffering data packets between the uplinks 40-55 and 
downlink 60. The scheduler 152 allocates bandwidth 
based upon a priorityclass packet service schedule 200 
calculated based upon traffic parameters associated 
with each active queue 104-110 (steps 306 and 308). 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention generally relates to a 
satellite communications network and more specifically 
to a satellite communications network for real time 
down-link scheduling of statistical bandwidth of data 
streams buffered through an out-bound module of a 
communications satellite. 

[0002] Satellite communications systems have been 
proposed for relaying communications signals between 
one or more user terminals and a ground station. The 
user terminals may be located at various places about 
the world with multiple terminals located in a common 
field of view of one satellite. The satellites typically in- 
clude antenna arrays that define spot beams or foot- 
prints on the earth's surface. The satellites generally re- 
ceive communications data streams from multiple ter- 
minals located in each spot beam. The data streams are 
received over separate uplinks or virtual channel con- 
nections. The satellite combines the incoming data 
streams for terminals in common spot beams and pass- 
es a combined down-link signal containing multiple data 
streams to a ground station or another user terminal. 
The receiving terminal or ground station may be located 
in a common spot beam as the transmitting terminal or 
in a different spot beam. 

[0003] Satellite communications networks are both in 
existence and under development for supporting com- 
munications channels carrying a variety of data types, 
such as audio data, video data, video conferencing data, 
broadcast video data, web surfing data and the like. Ex- 
isting satellite communications networks support sever- 
al connection-oriented protocols, such as the ATM 
(asynchronous transfer mode) protocol for conveying 
the various data types. As a connection oriented proto- 
col, an ATM network operates such that, before a call is 
permitted to take place through the network, a network 
provider guarantees that existing calls will not be unduly 
adversely affected. For instance, the network provider 
guarantees that existing connections will maintain a cer- 
tain minimum quality of service. In addition, before es- 
tablishing a new call within an ATM network, the network 
provider must insure that the network will afford a de- 
sired minimum quality of service for the new call. 
[0004] The quality of service defines an ATM require- 
ment for a particular connection, such as a data stream 
packet/loss ratio, a delay or delay variation between da- 
ta stream packet transmissions and the like. Two exem- 
plary ATM network control techniques are guaranteed 
minimum quality of service and best efforts minimum 
quality of service. In an ATM network, the quality of serv- 
ice for new calls and existing calls may be used for sev- 
eral functions, such as to determine call admission con- 
trol for new calls and scheduling with respect to the time 
of transmission for packets associated with existing 
calls. 



[0005] Call admission control is the process by which 
an ATM network determines whether or not to accept a 
new call or user who is requesting some service from 
the network. The freedom and flexibility afforded within 

5 call admission control and scheduling are dependent 
upon an amount of bandwidth afforded to an ATM net- 
work and upon the bandwidth requirements of individual 
calls requested from the network. Bandwidth in this con- 
text refers to a statistical amount of bandwidth required 

10 by a particular connection in order to meet a minimum 
quality of service required by the connection. 
[0006] Scheduling is the process by which the net- 
work determines the order in which data packets or cells 
are transmitted over a downlink to a ground or earth sta- 

*5 tion. In more detail, data packets or cells are continu- 
ously switched within the satellite among uplink chan- 
nels to associated out-bound modules. An out-bound 
module is associated with each down-link to a ground 
station or earth station. An out-bound module repre- 

20 sents a point in the satellite at which several traffic 
streams are combined into a common single stream. For 
instance, the out-bound module may be a module in the 
satellite in which multiple uplink traffic streams are com- 
bined and passed to a single down-link traffic stream. 

25 The uplink traffic streams may be for multiple users lo- 
cated in common or different footprints of the spot 
beams created by the satellite. A router or switch is pro- 
vided in the satellite for receiving incoming data packets 
from the different data streams. The router or switch 

30 identifies, for each data packet, a destination down-link 
beam. Based on the destination for each data packet, 
the router or switch directs each data packet to a corre- 
sponding out-bound module. 

[0007] Each out-bound module includes one or more 

35 queues or buffers which receive data packets associat- 
ed with a particular quality of service. The data packets 
stored in each buffer are output in a FIFO manner (first 
in, first out) or FIFS manner (first in, first served) . The - 
order between buffers is determined by the scheduler. 

40 [0008] The capacity of present microprocessor tech- 
nology has impacted the amount of processing that may 
be implemented on a satellite in a digital communica- 
tions network. Conventional ATM network switching el- 
ements require complex scheduling algorithms to be 

45 processed for quality of service and fairness guaran- 
tees. The limits of current processing speeds have ren- 
dered it difficult to permit real time processing associat- 
ed with quality of service and fairness guarantees, given 
that the typical network should perform scheduling cal- 

50 culations every few seconds for real-time scheduling. 
Current scheduling systems operate upon a static 
scheduling table that is defined based on a priori knowl- 
edge and historic traffic patterns. However, use of static 
scheduling tables leads to wasted bandwidth resources, 

55 unfair bandwidth allocation and ultimately loss of 
throughput and revenue. 

[0009] In the past, schedulers have been proposed 
that utilize a look-up table that stores a packet service 
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schedule identifying the order in which the buffers or 
queues output data packets or cells over the downlink. 
The packet service schedule defines a master frame 
comprised of data packets arranged in sequential time 
slots and passed over the downlink. The packet service 5 
schedule comprises a series of entries, each of which 
is associated with a time slot in the master frame. Each 
time slot in the look-up table stores a queue index iden- 
tifying a queue to be accessed during the corresponding 
time slot. Heretofore, schedulers have been prepro- 
grammed with packet service schedules storing static 
data transmission orders based on models of projected 
or estimated traffic demands. Conventional look-up ta- 
bles are periodically reprogrammed. However, when re- 
programmed, the models used to define the look-up ta- 
bles are based on a priori knowledge concerning historic 
and projected traffic patterns and bandwidth demands. 
[0010] However, scheduling based on a priori knowl- 
edge may be wasteful. First, a user may utilize less 
bandwidth than estimated. Consequently, a model may 
assign in the look-up table an excessive number of time 
slots to a queue having a particular quality of service, 
even though the channel requiring the particular quality 
of service does not need the estimated amount of band- 
width. In this manner, bandwidth in the down-link is as- 
signed, but remains idle. Alternatively, a channel may 
utilize more bandwidth than estimated. Excessive band- 
width usage requires additional resources to be called 
upon in order to service the particular user (e.g., an ad- 
ditional buffer may be used having a common or better 
quality of service than the buffer that is overloaded). 
[0011] A need remains for an improved downlink 
scheduler, it is an object of the preferred embodiments 
of the present invention to meet this need. 

BRIEF SUMMARY OF THE INVENTION 

[0012] A preferred embodiment of the present inven- 
tion utilizes a method for scheduling, in real-time, an or- 
der in which data packets from a plurality of uplink chan- 
nels are organized in a downlink of a satellite commu- 
nications network. The method includes conveying data 
packets over a downlink in an order determined by a 
packet service schedule. The method monitors at least 
one traffic parameter associated with each uplink chan- 
nel conveying a data stream. The traffic parameter is 
representative of an actual bandwidth usage of the cor- 
responding uplink channel. While conveying data pack- 
ets over the downlink, the method modifies the packet 
service schedule based on the monitored traffic param- 
eters. The actual bandwidth used by each uplink chan- 
nel is monitored to retain an accurate record of the de- 
mand being placed on each channel. Data packets are 
temporarily stored in corresponding queues in an out- 
bound module of the satellite. The data packets are di- 
rected to a corresponding queue by a switch based on 
service requirements associated with the channel car- 
rying such data packets. 



[001 3] Phase information may be measured for each 
channel. The phase information is indicative of an 
amount of time lapsed since a data packet from a par- 
ticular channel was last output from a queue to the 
downlink. The phase information is subsequently used 
when calculating the updated packet service schedule. 
New traffic parameters are continuously obtained for 
each channel by monitoring the data packets as they 
arrive at the queues associated with each channel. The 
data packets are switched from each channel to a 
unique queue or set of queues where the data packets 
are temporarily stored before being output to the down- 
link. 

[0014] Optionally, the packet service schedule may 
be stored in a look-up table. The processing associated 
with conveying data packets, monitoring traffic param- 
eters and modifying the packet service schedule based 
on the traffic parameters may all be carried out onboard 
the satellite. Alternatively, a portion of the calculations 
may be carried out at a ground station and the like. 
[001 5] A new packet service schedule is calculated 
based on the traffic parameters according to one of var- 
ious well known fair queuing algorithms, such as a Start- 
ing Potential Fair Queuing algorithm, a Frame Based 
Fair Queuing algorithm and the like. A dynamic amount 
of bandwidth is allocated for each channel in the packet 
service schedule. The allocated bandwidth for at least 
one channel is adjusted while the channel is conveying 
data packets in order to more effectively utilize the avail- 
able bandwidth. The allocated bandwidth is adjusted by 
modifying the packet service schedule to increase or de- 
crease the number of time slots assigned to a particular 
queue. 

[0016] An alternative preferred embodiment of the 
present invention includes a communications satellite 
comprising at least one uplink and a downlink for con- 
veying data packets over channels between terminals 
and ground stations. A switch directs data packets from 
each uplink channel to a unique queue. The satellite in- 
cludes a plurality of queues for collecting data packets 
from the uplinks and outputting the data packets to the 
downlink, wherein the amount of bandwidth associated 
with each queue is dynamically adjusted. A scheduler 
is provided in the satellite for allocating bandwidth. The 
scheduler changes the amount of bandwidth allocated 
to the queues while the queues are buffering data pack- 
ets between the uplinks and downlink. 
[0017] A bandwidth measurement module measures 
a statistical bandwidth actually being used by each 
queue. The scheduler updates the bandwidth allocated 
to each queue based on the measured statistical band- 
width. A look-up table may be used to store a packet 
service schedule defining a master frame allocating 
bandwidth to each active queue. The master frame com- 
prises a plurality of sequential time slots. The scheduler 
assigns to each time slot a priority queue index identi- 
fying a queue, from which a data packet should be out- 
put during the associated time slot in the master frame. 
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[0018] Optionally, the satellite may include a module 
for measuring the data packet rate for each queue. A 
processor in the scheduler calculates statistical band- 
width allocation for each queue based on the measured 
data packet rate. The scheduler further includes mem- 5 
ory storing the packet service schedule. The processor 
calculates a new bandwidth allocation for each queue 
based on one of several known fair queuing algorithms. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] Fig. 1 illustrates a satellite communications 
system supporting an ATM network in according with a 
preferred embodiment of the present invention. 
[0020] Fig. 2 illustrates an outbound module housed 
within a communications satellite in accordance with a 
preferred embodiment of the present invention. 
[0021] Fig. 3 illustrates a portion of an exemplary 
packet service schedule utilized in accordance with a 
preferred embodiment of the preferred embodiment of 
the present invention. 

[0022] Fig. 4 illustrates a flow chart outlining a 
processing sequence carried out in accordance with a 
preferred embodiment of the present invention. 
[0023] Fig. 5 illustrates a sequence of master frames 
assigned in accordance with a preferred embodiment of 
the present invention in real-time based upon traffic pa- 
rameters measured while the queues are in active use. 

DETAILED DESCRIPTION OF THE INVENTION 

[0024] Figure 1 illustrates a satellite communications 
system 5 that supports an ATM network for carrying 
communications data, such as voice, audio, video, vid- 
eo conferencing, broadcast video, internet information, 
data and the like. The satellite communications system 
5 includes a satellite 10 that communicates with one or 
more user terminals 15, 20, 25 and 30 via bi-directional 
communications links 40-55. The bi-directional commu- 
nications links 40-55 each include an uplink connection 
and a downlink connection (not separately shown) for 
passing communications data to and from the satellite 
10. The satellite 10 also communications with an earth 
station or gateway 35 and other user terminals (some 
of which are also transmitting on the uplink) via bi-direc- 
tional communications link 60 which includes an uplink 
and a downlink for passing communications data to and 
from the satellite 10. The user terminals 15-30 may be 
in a common footprint 65 of a single spot beam or many 
spot beams defined by an antenna of the satellite 10. 
The ground station 35 or other receiving user terminals 
may be in the same or a different footprint of a common 
or different spot beam. 

[0025] The bi-directional communications links 40-55 
may carry one or more data streams to and from the 
satellite 10. Each data stream supported by the bi-direc- 
tional communication links 40-55 corresponds to a sin- 
gle call or connection previously or to be established by 



the ATM network. By way of example only, user terminal 
15 may be a hand held mobile cellular phone that es- 
tablishes a connection with the ATM network by main- 
taining an uplink connection and a downlink connection 
over bi-directional communications link 40 with the sat- 
ellite 10. The uplink connection from the user terminal 
1 5 to the satellite 1 0 would include data transmitted from 
the user terminal 15, such as voice data. The satellite 
10 collects, via the uplinks within bi-directional commu- 
nication links 40-55, all data transmitted from user ter- 
minals 15-30. Upon receiving the uplink information 
from bi-directional communication links 40-55, the sat- 
ellite 10 combines the received data based upon ATM 
network protocols and transmits the combined data over 
a downlink in the bi-directional communications link 60 
to the ground station 35. The downlink is comprised of 
a data stream organized into consecutive similarly struc- 
tured master frames, each of which is divided into time 
slots. In the reverse direction, the satellite 10 receives 
uplink data from the ground station 35 and disseminates 
the uplink data to corresponding user terminals 15-30. 
[0026] Figure 2 illustrates a satellite outbound module 
100 according to a preferred embodiment of the present 
invention. The outbound module 100 represents a point 
in the satellite 10 where several uplink traffic streams 
are combined into a single stream to be passed over a 
downlink. For instance, the outbound module may be a 
module in the satellite in which multiple uplink traffic 
streams are combined and passed to a single downlink 
traffic stream. The uplink traffic streams may be for mul- 
tiple users located in common or different footprints of 
the spot beams created by the satellite 10. A router or 
switch is provided in the satellite for receiving incoming 
data packets from the traffic streams. The router/switch 
identifies, for each data packet, a destination downlink 
beam. Based on the destination for each packet, the 
router/switch directs each data packet to a correspond- 
ing outbound module 1 00. 

[0027] The outbound module 100 is housed within the 
satellite 1 0 and operates to receive communications da- 
ta over the uplinks within bi-directional communications 
links 40-55. The communications data received by the 
outbound module 100 is segmented into packets, such 
as in accordance with the ATM protocol. The outbound 
module 100 conveys the packets of communications da- 
ta, via queues or buffers, to a common downlink such 
as in bi-directional communications link 60. The aggre- 
gate queued communications data is subsequently 
passed to ground station 35. The outbound module 100 
includes a switch 102 for receiving communications da- 
ta from a plurality of individual connections or callers. 
Each connection or caller is associated with a single up- 
link from the user terminals 1 5-30 and a single queue 
for downlinking. 

[0028] By way of example, Figure 2 illustrates three 
incoming communications signals 130-132 which are 
subsequently redirected by the switch 102 to corre- 
sponding buffers 104-110. The buffers 104-110 may 
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vary in number and criteria for receiving communica- 
tions data packets. In the example of Figure 2, n buffers 
are illustrates, each of which has a different priority class 
or quality of service (QoS) associated therewith. For in- 
stance, buffer 104 may receive communications data 
packets from a data stream that require a particular 
quality of service designated as QoS1 . Similarly, buffers 
106 and 108 may receive communications data packets 
from corresponding data streams associated with sec- 
ond through fifth qualities of services QoS2-QoS5. 
[0029] The switch 102 directs corresponding packets 
of communications data from connections 130-132 to a 
corresponding buffer 104-110 based upon the quality of 
service required by the connection. 
[0030] The buffers 104-110 represent FIFO (first in 
first out) or FCFS (first come first served) queues, 
whereby an incoming data stream of packets is tempo- 
rarily stored or queued up in one of buffers 104-110 be- 
fore being selected for transmission to the downlink 
within bi-directional communications link 60. A schedul- 
er 1 52 determines the order in which the buffers 1 04-1 1 0 
distribute data packets to the downlink. 
[0031] The outbound module 100 further includes a 
bandwidth measurement module 114 which communi- 
cates with a packet monitor 116 in order to record and 
track the data streams passing over lines 120-126 
through buffers 104-110. The packet monitor 116 meas- 
ures at least one traffic parameter associated with each 
data stream carried over lines 120-126 and queued into 
the buffers 104-110. By way of example only, the traffic 
parameters may include one or more of a count of the 
number of packets received per second by each buffer, 
a count of the number of bits received per second by 
each buffer, a count of the total number of packets re- 
ceived by a particular buffer over a period of time and/ 
or a count of the total number of bits received by a par- 
ticular buffer over a period of time. 
[0032] The traffic parameters collected by the packet 
monitor 116 are passed to a bandwidth measurement 
module 114 which calculates a statistical bandwidth ac- 
tually being used by the data streams carried over paths 
120-126 and through buffers 104-110. The bandwidth 
measurement module 114 calculates statistical band- 
width for each data stream based upon the traffic pa- 
rameters collected by the packet monitor 116. The band- 
width measurement module 114 may calculate statisti- 
cal bandwidths for individual buffers 1 04- 110, for groups 
of buffers associated with a common priority class, for 
an entire outbound module 100 and the like. By way of 
example, the bandwidth measurement module 114 may 
calculate the aggregate arrival rate per outbound mod- 
ule 100 (e.g., mean, standard deviation and the like). 
The aggregate arrival rate per outbound module corre- 
sponds to the rate (packets or bits per second) at which 
the data streams are received. In addition, the band- 
width measurement module 114 may calculate the over- 
all inter-arrival time(IAT) per outbound module (e.g., 
mean, standard deviation and the like). The overall inter- 



8 

arrival time represents the time between arrival of pack- 
ets. The bandwidth measurement module 114 may cal- 
culate the overall inter-arrival time for the outbound 
module 100 in order to determine the burstiness of the 
5 data streams as a whole received by the outbound mod- 
ule 100. 

[0033] In addition or alternatively, the bandwidth 
measurement module 114 may calculate the statistical 
bandwidth per buffer or priority class. By way of exam- 

10 pie, the bandwidth measurement module 114 may cal- 
culate the arrival rate per buffer 1 04-1 1 0 and/or the inter- 
arrival time per buffer 1 04-1 1 0. The arrival rate and inter- 
arrival time per priority class may be characterized in 
any statistical fashion, such as by the average, mean, 

15 median, standard deviation and the like. Similarly, the 
statistical bandwidth for the outbound module 100 may 
be characterized by utilizing any statistical measure- 
ment, such as the average, mean, median, standard de- 
viation and the like. 

20 [0034] Once the traffic parameters are obtained, the 
statistical calculation may be made for each desired ran- 
dom variable in order to calculate the equivalent band- 
width for one or more data streams. The parameters for 
real-time statistical bandwidth calculation per priority 

25 class and per outbound module are, by way of example 
only, moving average estimation, standard deviation es- 
timation, and information vector (I = [X, a 2 , a]). Variables 
that may be used to calculate the real-time statistical 
bandwidth include u. which represents priority-class 

30 buffer index, which represents mean average arrival 
rate (packets/second), o^ 2 which represents variance of 
arrival rate (packets/seconds) 2 ), oc^ which represents in- 
stantaneous variance (per unit time) of the inter-arrival 
time packet arrival process (cells/second), IAT^ which 

35 represents mean inter-arrival time, and IAT^ 2 which rep- 
resents the second moment of the inter-arrival time. 
[0035] The schedule 152 includes a CPU 154 and a 
memory 156. The CPU 154 calculates the bandwidth 
allocation, in real-time, for each of queues 1 04- 1 1 0. The 

40 CPU 154 may utilize any of several well-known algo- 
rithms for calculating bandwidth allocation, such as 
Starting Potential Fair Queuing, Frame Based Fair 
Queuing and the like. Exemplary fair queuing algorithms 
are described in the following articles: 

45 

1) IEEE Communications Magazine, "Hardware im- 
plementation of Fair Queuing Algorithms for Asyn- 
chronous Transfer Mode Networks", Varma and Stil- 
iadis, Pg. 54 et seq (December 1999). 
50 2) Proceedings of the IEEE, "Service Disciplines for 
Guaranteed Performance Service in Packet- 
Switching Networks", Zang, Vol. 83, No. 10, Pg. 
1374 et seq (October 1999). 

3) Proceedings of the 1997 International Confer- 
55 ence on Networking Protocols, "High Speed, Scal- 
able, and Accurate Implementation of Packet Fair 
Queuing Algorithms in ATM Networks", Bennett, 
Stephens and Zhang, Pg. 7-14, (October 1997). 
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[0036] The CPU 154 generates a packet service 
schedule based on the fair queuing algorithm calcula- 
tions. The packet service schedule is stored in memory 
156. Optionally, the packet service schedule may be 
stored as a look-up table defining a master frame com- 5 
prised of time slots that set forth an order in which 
queues 104-110 output data packets. 
[0037] Fig. 3 illustrates an exemplary packet service 
schedule 200 comprised of multiple columns including 
a time index column 202, a relative time column 204 and 
a priority index column 206. The packet service sched- 
ule 200 is divided into rows. Each row corresponds to a 
time slot 208 in the master frame. In the example of Fig. 
3, time index 0 corresponds to relative time 0 and time 
slot 1 . Time slot 1 has been assigned to queue Q1 . Sim- 
ilarly, time indices 1, 2 and 3 indicate that queues Q2, 
Q3 and Qn have been allocated to the second, third and 
fourth time slots 210, 212 and 214. In the example of 
Fig. 3, each time slot 208 corresponds to a discreet pe- 
riod of time such as 2.88 microseconds. By way of ex- 
ample only, a master frame may be 30 milliseconds in 
length. In the example, the master frame would be di- 
vided into 10,418 time slots, each of which is 2.88 mi- 
croseconds in length. Each time slot 208 is assigned to 
one of queues Q1-Qn. The master frame represents 
100% of the statistical bandwidth that is available over 
a particular downlink. The scheduler controls the per- 
centage of bandwidth to be allocated to each of queues 
Q1-Qn. Based on these bandwidth percentage alloca- 
tions, the scheduler assigns each time slot to a particular 
queue Q1-Qn. For instance, if queue Q1 is allocated 
25% of the total bandwidth, then the CPU 1 54 will assign 
25% of the time slots 208 to queue Q1. Preferably, the 
time slots assigned to queue Q1 will be evenly distrib- 
uted throughout the master frame and packet service 
schedule 200. 

[0038] During operation, the scheduler 152 continu- 
ously obtains traffic parameter information from the 
bandwidth/phase measurement module 114 and based 
thereon continuously recalculates the bandwidth alloca- 
tion for each of queues Q1-Qn. When the traffic param- 
eters indicate that a particular queue is experiencing 
higher traffic than previously measured, the scheduler 
152 may increase the bandwidth allocated to the partic- 
ular queue while insuring a minimum quality of service 
required by all connections being passed through the 
queue and all other connections in the downlink. When 
increasing the bandwidth allocated to a particular queue 
in order to maintain that queues quality of service, the 
schedule 152 may reduce the bandwidth allocated to 
another queue which is using less bandwidth than was 
previously required to maintain its quality of service. 
[0039] Fig. 4 illustrates an exemplary processing se- 
quence carried out by the scheduler 1 52 in order to dy- 
namically and in real-time allocate bandwidth to any ac- 
tive channels passing through queues Q1-Qn. Initially, 
the scheduler 152 loads an initial packet service sched- 
ule into the look-up table 1 56 (step 300). Next, the proc- 



essor 154 obtains new cell counts and phase informa- 
tion associated with each of queues Q1-Qn from the 
bandwidth/phase measurement module 114 (step 302). 
Then the scheduler 152 begins servicing data packets 
stored in active queues Q1-Qn according to the initial 
packet service schedule (step 304). 
[0040] While the scheduler 152 services queues 
Q1-Qn according to the initial packet service schedule, 
the CPU 154 calculates a new packet service schedule 
based on the new cell counts and phase information ob- 
tained from module 114 (step 306). Finally, the CPU 154 
writes the new packet service schedule to the look-up 
table 156 (step 308). The process of steps 302-308 is 
continuously repeated throughout operation of the com- 
munications satellite network in order to continuously 
monitor the actual demand upon data channels passing 
through queues Q1-Qn. Based on the actual demand, 
the scheduler 152 is able to update, in real-time, the 
bandwidth allocation more efficiently and effectively. 
[0041] Fig. 5 illustrates an exemplary implementation 
of at least one preferred embodiment of the present in- 
vention. Fig. 5 illustrates three master frames 500-504 
associated with three separate packet service sched- 
ules stored in look-up table 156 at different points in 
time. Each of master frames 500-504 are generated by 
the CPU 154 during operation of the outbound module 
100 based upon the real-time traffic demands being 
placed upon queues Q1-Qn. Each master frame 
500-502 is comprised of a plurality of time slots 506. 
Each time slot 506 designates a particular queue Q1 -Qn 
to output a data packet during the corresponding time 
slot in the downlink. In the example of Fig. 5, master 
frame 500 may represent an initial demand placed upon 
the outbound module 100. During the initial demand as- 
sociated with master frame 500, only queues Q1 and 
Q2 carry traffic, while the remaining queues Q3-Qn re- 
main idle (inactive). 

[0042] Returning to Fig. 4, at step 300, the CPU 1 54 
loads the initial packet service schedule associated with 
master frame 500 into the look-up table 156. The initial 
packet service schedule may be generated by the CPU 
154. Alternatively, the initial packet service schedule 
may be uploaded from a ground station. During the initial 
service, while queues Q1 and Q2 are the only active 
queues, the scheduler 1 52 may allocate the entire band- 
width available through the outbound module 100 to 
queues Q1 and Q2. When it is desirable to allocate the 
entire bandwidth to only the active queues Q1 and Q2, 
the master frame 500 is loaded such that all of the time 
slots 506 are assigned to one of queues Q1 and Q2. 
[0043] In the example of Fig. 5, it is assumed that the 
minimum quality of service required by Q1 corresponds 
to at least 10% of the entire bandwidth available to out- 
bound module 1 00, while the minimum quality of service 
required by the channel associated with queue Q2 re- 
quires at least 20% of the total available bandwidth over 
outbound module 100. As the remaining queues Q3-Qn 
are idle, additional bandwidth in excess of the minimum 
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requirements has been afforded to queues Q1 and Q2 
in the master frame 500. The total bandwidth assigned 
to queues Q1 and Q2 may be proportional to the relation 
of the minimum qualities of service for each queue. Con- 
sequently, as queue Q2 requires a quality of service 
having approximately double the bandwidth as com- 
pared to the quality of service associated with queue Q1 , 
twice as many time slots 506 in master frame 500 may 
be assigned to queue Q2. In the example of Fig. 5, the 
time slots are assigned such that two time slots are as- 
signed to queue Q2 for every time slot assigned to 
queue Q1. 

[0044] After the initial packet service schedule corre- 
sponding to master frame 500 is loaded in step 300 and 
used by the scheduler 152, processing passes to step 
302 (Fig. 4) at which the scheduler 1 52 obtains new cell 
counts and phases from the queues Q1-Qn. Control 
passes to step 304 where the scheduler 152 begins 
servicing cells Q1 and Q2 according to the packet serv- 
ice schedule designated in master frame 500. 
[0045] Next, the scheduler 152 calculates a new 
packet service schedule based on the new cell counts 
and phases obtained in step 302. At step 306, in the 
example of Fig. 5, it is assumed that queue Q3 becomes 
active and thus requires some portion of bandwidth al- 
location. In the example of Fig. 5, the bandwidth/phase 
measurement module 114 measures the traffic param- 
eters associated with queue Q3 and passes these traffic 
parameters to the scheduler 152. The processor 154 
calculates from the traffic parameters for queue Q3 that 
the minimum quality of service required by the actual 
demand being placed on queue Q3 requires approxi- 
mately 10% of the total bandwidth available in the out- 
bound module 100. Consequently, the processor 154 
reallocates the time slots 506 in the master frame 502 
to afford at least a minimum bandwidth to queue Q3. 
[0046] Optionally, the processor 154 may assign the 
minimum number of time slots 506 to queue Q3 in order 
to afford 10% of the total available bandwidth to queue 
Q3. Alternatively, the queue Q3 may be afforded more 
bandwidth than its minimum requirement. In the exam- 
ple of Fig. 5, the processor 1 54 has determined from the 
traffic parameters for queues Q1-Q3 that the minimum 
bandwidth requirements for each queue are 10%, 20% 
and 10%, respectively, of the total available bandwidth. 
The processor 154 divides the total bandwidth of the 
outbound module proportionally between queues 
Q1-Q3. In other words, the processor 154 assigns 25% 
of the total bandwidth to queue Q1, 50% of the total 
bandwidth to queue Q2, and 25% of the bandwidth to 
queue Q3. These assignments are illustrated in master 
frame 502 by consecutively assigning time slots 506 in 
the following manner: Q1, Q2, Q2, Q3. 
[0047] Returning to Fig. 4, the scheduler 152 writes 
the new packet service schedule associated with the 
master frame 502 to the look-up table 156 and the 
scheduler 1 52 begins controlling queues Q1 -Q3 accord- 
ingly. Processing returns to step 302 at which the sched- 



uler 152 obtained new traffic parameters for queues 
Q1-Qn. In the example of Fig. 5, it is assumed that 
queues Q4, Q7 and Q16 become active and the asso- 
ciated traffic parameters indicate that the present band- 
5 width demands of queues Q4, Q7 and Q1 6, each require 
approximately 20% of the total available bandwidth from 
the outbound module 100. 

[0048] At step 304, the scheduler 152 begins servic- 
ing the cells in queues Q1-Q3 according to the master 

10 frame 502 stored as the packet service schedule. At 
step 306, the processor 154 generates master frame 
504 based on the new traffic parameters indicating that 
queues Q1, Q2, Q3, Q4, Q7, Q16 require 10%, 20%, 
10%, 20%, 20%, 20%, respectively, of the total band- 

15 width available in outbound port 1 00. The processor 1 54 
assigns the time slots 506 in master frame 504 as indi- 
cated in Fig. 5 as follows: Q1, Q2, Q2, Q3, Q4, Q4, Q7, 
Q7, Q16 and Q16. The foregoing sequence of queue 
assignments is continuously repeated throughout the 

20 length of the master frame 504. The master frame 504 
is written to the look-up table 156 in step 308 and the 
above process is repeated. In the foregoing manner, the 
scheduler 154 is able to maximize the use of the band- 
width available in outbound module 100. 

25 [0049] A fair queuing scheduling algorithm may be im- 
plemented using ground based processing depending 
upon the complexity of the algorithm desired to be used. 
It is preferable that ground based processing be mini- 
mized or eliminated as ground base processing requires 

30 uplink and downlink data resources to convey schedul- 
ing information to and from the satellite. Ground based 
processing table entries may arrive delayed at the com- 
munications satellite by several hundreds of millisec- 
onds due to propagation delay and input/output bottle- 

35 necks. Accordingly, it is preferable that scheduling op- 
erations be carried out on-board the satellite. 
[0050] Various scheduling algorithms differ in com- 
plexity, in processing resource demands and in process- 
ing time. An optimal fair queuing technique may require 

40 more processing resources (e.g., memory, speed, time) 
than are available on certain communications satellites. 
Accordingly, it may be preferable to load into an on- 
board scheduler multiple fair queuing algorithms, includ- 
ing at least one optimal algorithm and at least one sub- 

45 optimal algorithm. Additional fair queuing algorithms 
may be loaded on the satellite offering a range of accu- 
racy and precision. A sub-optimal algorithm may ap- 
proximate certain fair queuing characteristics. The sat- 
ellite may then be afforded the option of automatically 

so (or based on ground control) picking a fair queuing al- 
gorithm from multiple algorithms that best satisfies the 
implementation criteria of a particular application. 
[0051] Optionally, the frequency with which the traffic 
parameters for queues Q1-Qn are monitored may be 

55 varied (automatically or by ground control). The traffic 
parameter monitoring rate may be minimized, so long 
as sufficient information is made available to the CPU 
1 54 to create a packet service schedule 200 containing 
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time slot entries which substantially approximate a fair 
queuing algorithm. To further improve the processing 
speed, the scheduler 152 may be limited to one time 
stamp per queue (retained over time), one buffer occu- 
pancy statistic per queue, one fixed average service rate 5 
per queue and the like. 

[0052] When the I/O transmission rate to and from the 
scheduler 152 is unduly limited, the rate at which table 
entries may be programmed in the packet service 
schedule 200 may become slower than the downlink 
transmission rate. A ratio may be defined between the 
downlink transmission rate and the average rate at 
which time slots are updated in the look-up table. This 
ratio may be defined as the downlink scheduling per- 
centage. When the downlink scheduling percentage is 
unduly small, table entries become imprecise. The 
scheduler 152 becomes unable to update the table fast 
enough to retain precision, and thus must rely on ap- 
proximate service needs. A packet service schedule 
that has not been entirely updated, before all of the time 
slots are cycled through will be looped through and 
some queues will be given unnecessary bandwidth as 
time slots associated with the queues will not yet have 
been updated to reflect the current traffic parameter 
measurements. 

[0053] In addition, when the downlink scheduling per- 
centage becomes unduly small, the time stamps be- 
come unaccurate for queues. When the time stamps be- 
come inaccurate, the scheduler 152 is afforded with lim- 
ited knowledge of the output module 100 state and may 
be unable to maintain queue specific state information 
on a call-by-call basis. Consequently, when the down- 
link scheduling percentage becomes unduly small, it is 
preferable that the scheduler 152 approximate time 
stamps based upon the assumption that the majority of 
the queues are busy. In this instance, fair queuing guar- 
antees cannot be insured. 

[0054] However, during approximate fair queuing, ac- 
curacy is more important than precision. Buffer occu- 
pancy statistics (traffic parameters) fed to the scheduler 
1 52 enables estimates to be made of total queue occu- 
pancy. Consequently, during approximate fair queuing, 
static priority may be used by the scheduler 152 in ad- 
dition to the call service schedule 200 to service the 
queues 104-110 when idle queues are indexed by the 
packet service scheduler 200. In this instance, maps 
read from the NOC are no longer needed. 
[0055] For true fair queuing, when processing capa- 
bilities exceed or meet traffic demands, true (non-pseu- 
do) fair queuing may be achieved with only a slight in- 
crease in delay bounds. In this scheme, fair queuing cal- 
culated statistics (time stamps) match fairly closely with 
ongoing transmission and arrival events. Undue late ar- 
rival of traffic parameters to the scheduler 1 52 create 
the potential that table entries created from specific time 
slots 208 may not lead to service for the intended data 
packet. However, although some queues. 104-110 may 
perform bandwidth grabbing, the worse case delay im- 



posed on any one queue 104-110 as a consequence is 
accounted for in the delay bound increase. Consequent- 
ly, approximate onboard fair queuing is preferable to 
ground processed table updates. 
[0056] If the average table write rate is larger than the 
downlink transmission rate, it is preferable for true fair 
queuing that only so many entries as can be serviced 
are written. This circumstance occurs when the down- 
link scheduling percentage is less than one. The reason 
for this requirement is to avoid confusion. The scheduler 
152 checks off service given to all entries it programs. 
If entries are not reached, their queues may lose serv- 
ice. It is also important for true fair queuing that downlink 
scheduling percentages be greater than CAC average 
downlink utilization. The scheduler may only guarantee 
fairness over the portion of the downlink that it controls. 
[0057] For true fair queuing, precision is more impor- 
tant than accuracy in queue occupancy counters. 
Queue occupancy counters sent to the controller may 
have conceivably fewer bits than the full 16 bits availa- 
ble. In this case, true fair queuing requires one-to-one 
correspondence between counters and queue occu- 
pancies for low values. Buffer occupancies larger than 
the maximum expressably for the given number of bits 
are rounded down to the maximum. 
[0058] While particular elements, embodiments and 
applications of the present invention have been shown 
and described, it will be understood, that the invention 
is not limited thereto since modifications may be made 
by those skilled in the art, particularly in light of the fore- 
going teachings. It is, therefore, contemplated by the ap- 
pended claims to cover such modifications as incorpo- 
rate those features which come within the spirit and 
scope of the invention. 



Claims 

1. A method for scheduling, in real-time, an order in 
which data packets from a plurality of uplink chan- 
nels stored in priority-class queues are organised 
in a downlink channel of a satellite communications 
network, the method comprising: 

conveying data packets over a downlink chan- 
nel in an order determined by a packet service 
schedule; 

monitoring at least one traffic parameter asso- 
ciated with at least one data stream stored in a 
priority-class queue, the traffic parameter being 
representative of an actual bandwidth usage of 
the corresponding priority class queue; and 
while conveying data packets over the downlink 
channel, modifying the packet service schedule 
based on said at least one traffic parameter. 

2. The method of claim 1 , further comprising: 

monitoring an actual bandwidth used by each 
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priority-class queue. 

3. The method of claim 1, further comprising: 

measuring a phase of each data stream 
stored in a priority-class queue, said phase being 
indicative of an amount of time lapsed since a data 
packet from a particular priority-class queue was 
output to the downlink channel. 

4. The method of claim 1 , further comprising: 

calculating a new packet service schedule 
based on the traffic parameters according to a 
Packet Fair Queuing (PFQ) algorithm. 

5. The method of claim 1 , further comprising: 

allocating in the packet service schedule a dy- 
namic amount of bandwidth to each priority-class 
queue. 

6. The method of claim 1 , further comprising: 

modifying the packet service schedule by ad- 
justing an amount of bandwidth allocated to at least 
one priority-class queue while the priority-class 
queue is storing data packets. 

7. A communications satellite, comprising: 

at least one uplink and downlink for conveying 
data packets over communications channels; 

queues for collecting data packets from uplinks 
and outputting the data packets to a downlink 
using a dynamic amount of bandwidth; and 
a scheduler for allocating bandwidth to at least 
one queue, said scheduler changing an amount 
of bandwidth allocated to at least one queue 
while said queue is buffering data packets be- 
tween an uplink and downlink. 

8. The communications satellite of claim 7 further 
comprising: 

a bandwidth measurement module for meas- 
uring a statistical a statistical bandwidth actually be- 
ing used by at least one queue, said scheduler up- 
dating the bandwidth allocation of said at least one 
queue based on said measured statistical band- 
width. 

9. The communications satellite of the claim 7, further 
comprising: 

means for measuring data packet rate for 
each queue, said scheduler modifying bandwidth 
allocation based on the measured data packet rate. 

10. The communications satellite of claim 7, further 
comprising: 

means for monitoring at least one traffic pa- 
rameter associated with each downlink stream, said 
traffic parameter being representative of an actual 



usage of a priority-class associated with a queue, 
the scheduler changing bandwidth allocation based 
on said traffic parameter. 
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(54) Method and apparatus for real time scheduling in a satellite communications network 



(57) A method and apparatus are provided for 
scheduling, in real-time, the order in which data packets 
from a plurality of uplink channels 40-55 are organized 
in a downlink channel 60 of a satellite communications 
network 5. The satellite 10 includes uplink and downlink 
channels for conveying data packets over channels be- 
tween user terminals 1 5-30, ground stations 35 and oth- 
er user terminals. Queues 1 04- 1 1 0 in the satellite 1 0 col- 
lect data packets from uplinks 40-55 and output the data 



packets to the downlink 60 using a bandwidth that is dy- 
namically allocated. A scheduler 152 allocates the 
bandwidth to at least one queue 104-110 and continu- 
ously changes the amount of bandwidth allocated to 
each active queue 104-110 while the queue 104-110 is 
buffering data packets between the uplinks 40-55 and 
downlink 60. The scheduler 152 allocates bandwidth 
based upon a priori tyclass packet service schedule 200 
calculated based upon traffic parameters associated 
with each active queue 104-110 (steps 306 and 308). 
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